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PREFACE 


The  present  effort  was  conducted  in  support  of  the  Armstrong 
Laboratory/Aircrew  Training  Research  Division  (AL/HRA)  research 
concerning  image  quality  in  simulator  displays.  The  goal  of  this 
effort  was  to  develop  a  quantitative  model  which  predicted  display 
quality  based  upon  the  display  modulation  transfer  function  and 
display  luminance.  The  project  was  conducted  under  Work  Unit 
1123-03-85,  Flying  Training  Research  Support.  Research  support  was 
provided  by  the  University  of  Dayton  Research  Institute  under 
contract  No.  F33615-90-C-0005 .  The  contract  monitor  was  Ms. 
Patricia  Spears,  AL/HRAP. 

The  goal  of  this  specific  research  effort  was  to  (a)  initiate 
an  experimental  design  and  methodology  whereby  image  quality  of 
visual  displays  could  be  studied  in  a  multidimensional  environment 
and  (b)  collect  data  and  develop  a  multidimensional  model  to 
predict  image  quality  as  a  function  of  multiple  display  attributes. 

The  author  wishes  to  express  thanks  to  Ms.  Marge  Keslin  for 
final  edit  of  the  manuscript. 
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AM  EMPIRICAL  APPROACH  TO  VISUAL  DISPLAY  PREFERENCE  BASED 
UPON  MODULATION  TRANSFER  FUNCTION  AMD  LUMINANCE 


INTRODUCTION 

The  importance  and  prevalence  of  electronic  visual  displays  in 
society  continue  to  grow  as  technologies  (e.g.,  fiber  optics, 
microchip  and  compact  disk  technology)  improve.  As  computational 
power  and  the  capability  to  store  and  transmit  dynamic  visual 
imagery  improve,  the  use  of  electronic  displays  will  increase. 
Given  the  variety  of  tasks  for  which  humans  interact  with  visual 
displays  and  the  economics  involved  in  these  emerging  technologies, 
the  need  to  develop  methods  for  assessing  the  quality  of  imagery 
and  acceptable  standards  becomes  a  priority. 

The  Assessment  of  Display  Quality  as  a  Complex  Task 

Electronic  visual  displays  vary  along  a  number  of  dimensions, 
including  luminance,  resolution,  contrast,  display  size,  color,  and 
update  rate.  When  comparing  the  quality  of  two  displays,  it  may  be 
necessary  to  compare  multiple  dimensions  or  attributes  listed 
above.  In  many  instances,  improvement  along  one  dimension  is  often 
accompanied  by  a  loss  in  another  dimension.  For  example,  increases 
in  display  size  or  field  of  view  are  typically  accompanied  by  a 
drop  in  luminance.  Systems  with  greater  luminance  capabilities  may 
exhibit  a  drop  in  their  resolution  at  high  spatial  frequencies. 
Such  trade-offs  occur  rather  frequently  with  electronic  visual 
displays.  These  trade-offs  will  often  make  choices  unclear.  For 
example,  larger  displays  viewed  from  farther  away  may  have  more 
resolution  but  less  brightness  than  smaller  displays. 

Figure  l  shows  the  Michelson  Contrast  as  a  function  of  spatial 
frequency,  often  simply  referred  to  as  the  display  modulation 
transfer  function  (MTF) ,  for  two  displays  used  at  the  Aircrew 
Training  Research  Division  of  Armstrong  Laboratory  in  Mesa,  AZ. 
Functions  based  upon  t.ne  height  and  area  under  the  MTF  curve  often 
serve  as  image  quality  metrics  (e.g.,  Barten,  1989,  1991;  Evans, 
1990)  .  Note  how  the  curves  of  the  two  displays  cross  over  at 
approximately  2-3  cycles  per  degree  on  the  x-axis.  Using  only 
these  two  curves  for  comparison,  it  might  be  difficult  to  determine 
which  display  is  preferred.  Although  the  limited  f ield-of-view 
display  (LFOYD)  is  physically  larger,  the  display  for  advanced 
research  and  training  (DART)  is  much  brighter  and  is  typically 
preferred . 

Referring  to  Figure  1  as  an  example,  the  display  MTF  is  a 
measure  of  Michelson  contrast  (i.e.,  (Lmax-Lmin) / (Lmax+Lmin)  where 
Lmax  and  Lmin  denote  the  maximum  and  minimum  luminance  measured  on 
the  device)  capability  as  a  function  of  spatial  frequency.  As 
such,  the  MTF  is  a  multidimensional  display  factor  in  its  own 
right.  In  the  development  of  image  quality  metrics,  however,  the 
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display  MTF  has  been  the  ri'ost  often  used  display  factor  to  date. 
For  exaitiple,  in  Figure  1  a  simple  metric  would  be  to  compute  the 
area  under  each  curve  as  a  measure  of  display  quality .  More 
advanced  metrics  (e.g.,  Barten,  1990;  Snyder,  1985)  weight  the 
distance  along  the  x-axis  differentially.  Using  Barter's  square 
root  integral  (SQRI)  metric,  for  example,  the  DART  Display  in 
Figure  1  would  be  judged  to  be  superior  because  the  integration  of 
the  MTF  is  done  with  respect  to  the  log  of  the  spatial  frequency, 
thereby  emphasizing  the  lower  spatial  frequencies.  By  simply 
observing  the  two  display  MTFs  in  Figu'r’e  1,  however,  there  appears 
to  be  no  clear-cut  winner.  The  display  factor  of  luminance  plays 
a  much  greater  role  in  this  comparison. 
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Because  of  trade-offs  similar  to  those  discussed  above ,  a 
multidiinensional  approach  to  assessing  image  quality  is  required. 
Here  multiple  display  factors  are  manipulated  and  observer 
preference  or  pet  formai'ice  is  measured.  Such  an  approach  allov;s  us 
to  relate  changes  across  dimensions  of  interest  (i.e.,  luminance, 
display  size)  by  their  effect  "^n  the  dependent  variable  in  the 
experiment . 


There  are  problems  with  using  sucli  an  approach,  how'ever.  From 
a  hardware  view'point,  it  is  quite  difficult  to  manipulate  display 
factors  or  parameters  independently  of  one  another.  If  image 
quality  is  compared  across  tV'.*c  display  systems,  the  difficulty  is 
in  manipulating  the  factors  of  interest  while  holding  other  display 
factors  equivalent  across  the  two  displays.  The  alternative  used 
in  the  present  study  is  to  use  a  single  display  and  manipulate 
display  parameters  within  the  single  display.  As  will  be  seen  in 
the  present  study,  use  of  a  single  display  limits  the  range  of  the 
variables  manipulated  in  the  study. 


In  the  present  study,  :  factors  have  been  chosen  to  be 
manipulated.  These  are  the  ^..^cplay  MIT  (actually,  the  area  under 
the  MTF  curve)  and  the  display  luminance.  The  goal  of  the  present 
study  is  to  develop  a  prediction  scheme  for  display  preference  as 
a  function  of  display  MTF  and  luminance.  Figure  2  portrays  a  plane 
or  c’wo-dimensional  graph  where  display  MTF  and  luminance  represent 
the  X  and  y  axes.  If  all  displays  can  be  character ized  as  points 
in  this  plane,  then  a  third  orthogonal  dimension  can  be  used  to 
represent  preference.  Note  that  in  Figure  2  both  MTF  and  luminance 
are  unidimensional  quantities.  For  the  preseiit  study,  the  area 
under  the  display  MTF  will  be  used  to  represent  the  unidimensional 
MTF  quantity  and  average  display  luminance  will  be  used  to 
represent  luminance.  Both  of  these  definitions  are  simplifications 
of  the  actual  concepts  concerning  contrast,  resolution,  and 
brightness  within  imagery . 

Up  to  this  point,  the  discussion  of  display  quality  has 
focused  only  upon  the  characteristics  of  the  display.  Figure  3 
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Figure  2 

imensional  Plot  of  MTF  and  Luminance  Combinations 
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Figure  3 

Systems  Approach  to  Image  and  Display  Quality 


portrays  a  systems  approach  to  image  quality  which  emphasizes  (a) 
the  characteristics  ci  the  image  which  are  fed  to  the  display,  (b) 
the  effect  of  the  display  system  on  the  imagery,  and  (c)  the 
perceptual  processing  and  limitations  of  the  observer.  In  Figure 
3,  these  three  components  are  contained  within  a  box  defined  as  the 
task  domain.  Theoretically,  the  quality  of  a  display  could  be 
determined  for  asymptotic  levels  of  (a),  (c)  ,  and  the  task  domain. 
For  the  purpose  of  procuring  displays  for  specific  functional 
tasks,  though,  the  quality  of  a  display  cannot  be  ascertained 
independently  of  components  (a)  ,  (c)  ,  or  the  task  domain.  For 
example,  consider  two  displays  which  are  equivalent  except  that  one 
display's  limiting  resolution  is  25  cycles/degree  while  the  other's 
limiting  resolution  is  30  cycles/degree.  If  the  imagery  fed  to  the 
display  is  limited  to  20  cycles/degree,  the  two  displays  will 
appear  to  be  equivalent  for  all  tasks.  As  in  any  practical  linear 
system,  the  limitation  of  the  weakest  component  of  the  system 
becomes  a  limiting  factor  in  the  system's  performance.  The  entire 
system  is  included  witiiin  the  task  domain  box  as  a  reminder  that 
the  imagery  or  information  at  the  front  end  of  tlie  system  and  the 
processing  at  the  observer  end  will  typically  be  task  specific. 

•An  important  concept  in  the  systems  approach  from  Figure  3  is 
that  the  information  contained  in  the  original  image  affects  all 
subsequent  processes.  Because  of  this  and  the  inherent  variability 
in  and  between  images,  several  images  will  be  used  in  the  present 
study . 


Along  with  the  systems  approach,  which  emphasizes  the 
sequential  nature  and  influence  of  components  on  one  another,  the 
notion  of  image  quality  must  be  operationalized  in  order  to  be 
studied  in  a  scientific  manner.  Operational  definitions  of  imago 
quality  typically  rely  on  human  response  or  performance  which 
hypothetically  varies  as  a  function  of  the  display  manipulations. 
In  a  majority  of  previous  experimental  studies,  two  task  types  have 
been  primarily  used  in  defining  image  quality.  In  the  first  type 
of  task,  observers  rate  imagery  from  displays  based  upon  aesthetic 
aualities  (e.g.,  sharpness  with  Ku-saka,  1989).  This  method  of 
defining  image  quality  may  involve  global  interpretation  (i.e., 
based  upon  a  large  composite  of  display  factors)  by  the  observer 
and  can  bo  quite  subjective.  The  second  type  of  task  uses 
norformance  measures  such  as  target  detection,  recognition,  and 
identification  (e.g.,  Snyder,  1985).  In  these  tasks,  improvements 
in  performance  would  be  interpreted  as  an  improvement  in  image 
quality.  Manipulations  in  rating  tasks  are  usually  easily 
detectable  and  the  experimenter  is  not  interested  in  working  near 
the  bounds  of  the  observer's  perceptual  limits.  Performance  tasks 
typically  employ  threshold  manipulations  of  factors  in  order  to 
produce  errors  or  have  some  deleterious  effect  on  performance. 

In  the  current  study,  a  paired-comparisons  rating  task  has 
been  chosen  as  the  dependent  measure.  On  individual  trials, 
observers  will  be  presented  v.’ith  two  images  simultaneously,  both 
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images  being  filtered  by  two  of  a  number  of  MTF-luminancc 
combinations.  The  images  will  be  identical  except  for  the 
application  of  the  two  filters.  Observer  responses  will  denote 
which  of  the  two  images  is  preferred.  Referring  back  to  Figure  2, 
the  purpose  of  obtaining  the  paired-comparison  preferences  will  be 
to  generate  points  in  a  third  dimension  denoting  preference  for 
each  MTF-luminance  combination.  Modeling  will  be  used  to  generate 
these  preferences  from  the  paired-comparison  preferences  and 
regression  will  be  used  to  generate  a  preference  surface  in  the 
three-dimensional  space. 

Twenty  points  (5  MTF  levels  x  4  luminance  levels)  will  be 
generated  in  the  plane  of  Figure  2.  Using  the  paired-comparisons 
procedure,  there  v/ill  be  estimates  for  the  preference  of  any  of  the 
20  points  (MTF-luminance  combinations)  over  any  of  the  20  points 
(400  possiolo  comparisons).  Included  in  those  comparisons  is  the 
pairing  of  a  point  with  itself.  This  condition  serves  as  an 
estimate  of  noise  or  observer  sensitivity. 

As  mentioned  previously,  the  MTF  and  luminance  filtering  of 
the  imagery  is  a  technically  difficult  task.  In  this  study,  both 
images  are  presented  side-by-side  on  a  single  display.  The  display 
itself  serves  as  a  limitation  in  the  filtering  process.  The 
following  section  describes  how  MTF  and  luminance  filtering  were 
accomplished  for  the  study. 


TRANSFORMATION  OF  IMAGE  MTF  AND  LUMINANCE 

In  order  to  create  the  20  levels  of  imagery  (5  MTFs  X  4 
luminance  levels)  required  in  the  experimental  design,  it  was 
necessary  to  digitally  filter  each  of  the  five  images  by  each  of 
the  20  filters.  Before  discussing  hov/  tlie  filtering  was 
accomplished,  a  short  introduction  to  the  original  nature  of  the 
five  digital  images  is  in  order.  It  is  emphasized  here  again  that, 
as  shown  in  Figure  3,  the  characteristics  of  the  original  imago 
play  an  integral  role  in  any  experimental  results  obtained.  The 
effect  of  any  of  the  three  components  in  Figure  3  on  the  quality  of 
the  final  perception  cannot  be  interpreted  in  isolation  of  the 
remaining  two  components. 

Photographs  of  the  images  were  digitally  scanned  into  512  rows 
X  512  columns  of  Digital-to-Analog  Code  (DAC)  values.  Each  of  the 
512  X  512  DAC  values  could  take  any  value  between  0  and  255  (an 
eight-bit  DAC  value).  Figure  4  shows  a  frequency  distribution  of 
DAC  values  for  each  of  the  five  images  (airport  scene,  crop  scone, 
mountain  scene,  ocean  scene,  and  pines  scene).  The  y-axis  in  this 
graph  i.s  a  logarithmic  transformation  of  frequency.  Note  that  cill 
the  images  contain  an  abundance  of  low  luminance  (low  DAC  values) 
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Distribution  of  DAC  Values  for  Five  Images 


pixels  as  well  as  an  abundance  of  pixels  with  high  luminance  (DAC 
value  of  255) .  The  importance  of  the  frequency  distributions  in 
this  graph  will  be  explained  in  the  section  covering  luminance 
filtering  of  the  images.  Figure  5  shows  a  plot  of  the  gamma  curve 
for  the  display  used  in  the  experiment.  This  curve  relates  the  DAC 
value  of  a  pixel  in  the  image  to  the  luminance  (in  fL)  from  the 
display  used  in  the  experiment  (as  measured  with  a  photometer) . 
Two  curves  are  plotted  in  Figure  5.  The  upper  or  higher  curve 
shows  luminance  measured  from  the  display  as  a  function  of  DAC 
value  measured  from  a  small  area  in  the  center  of  the  display  when 
the  surrounding  area  was  completely  black  (BACKGROUND  DAC  =  0) . 
The  lower  curve  shows  similar  measurements  when  the  surrounding 
area  was  made  as  bright  as  possible  (BACKGROUND  DAC  =  255) .  Note 
that  very  little  variation  in  luminance  occurs  below  a  DAC  value  of 
50,  essentially  making  this  range  of  DAC  values  (0-50)  unusable  as 
a  source  of  luminance  variation.  A  power  curve  was  generated  which 
fit  between  the  two  curves  in  Figure  5  for  D.^C  values  in  the  range 
of  50  to  255.  The  equation  for  this  curve  is  as  follows: 

Luminance  (fL)  =  .00043  X  (DAC  VALUE)^  (1) 

Equation  (1)  and  the  curves  in  Figure  5  are  completely  dependent 
upon  the  brightness  setting  of  the  display.  The  brightness  control 
of  the  display  was  held  constant  throughout  the  experiment,  and  the 
only  purpose  in  the  use  of  the  curves  and  Equation  (1)  was  to 
relate  the  digital  representation  of  the  images  to  the  luminance 
produced  by  the  display  CRT. 

Beyond  specifying  the  luminance  profile  of  the  images  used  in 
the  experiment,  another  measure  often  used  to  describe  imagery  is 
the  spatial  frequency  content.  Figure  6  shows  a  one-dimensional 
(across  rows)  global  Fourier  analysis  of  the  digitized  images.  As 
will  be  the  case  with  most  natural  imagery  (e.g.,  Evans,  1993),  the 
shape  of  the  curves  from  Figure  6  are  highly  similar  and  provide 
little  discriminable  information.  A  Fourier  analysis  of  local 
areas  in  these  images  can  yield  curves  which  are  highly  dissimilar 
but,  when  averaging  over  large  spatial  areas  of  natural  imagery, 
the  low  frequency  components  will  dominate  the  Fourier  analysis  in 
Figure  6  (denoting  large  spatial  areas  of  little  luminance 
variation)  . 

The  five  images  used  in  the  experiment  are  shown  in  Figures  7 
through  11.  The  images  represent  an  airport  scene  (Figure  7),  a 
crop  scene  (Figure  8),  a  desert  mountain  scene  (Figure  9),  an  ocean 
scone  (Figure  10),  and  a  hilly  pine  scene  (Figure  11).  Note  the 
distinctiveness  of  the5;e  scenes  as  contrasted  with  the  lack  of 
distinction  in  the  global  fourier  analysis  of  the  imagery  (Figure 
6).  The  following  two  sections  describe  the  MTF  filtering  and  the 
luminance  filtering  applied  to  the  images  used  in  the  experiment. 
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Figure  5 

Gamma  Curve  for  the  Experimental  Display 
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Figure  6 

One-Dimensional  Fourier  Decomposition  of  Experimental  Imagery 


Digital  MTF  Filtering  of  iTnages 


The  purpose  of  the  MTF  filtering  of  the  images  is  to  simulate 
the  effect  of  a  range  of  display  MTFs.  In  order  to  do  this,  th.e 
simulated  MTFs  must  (a)  look  similar  so  actual  display  MTFs  and  (b) 
be  mathematically  tractable  in  order  to  create  and  employ  tlie 
filters  on  a  computer  for  digital  processing.  A  Gaussian  filter  is 
especially  attractive  for  this  purpose  because  it  has  been  linked 
to  CRT  display  MTFs  through  physical  measures  (e.g.,  Barten,  1984) 
and  its  representation  in  the  spatial  and  frequency  domains  are 
relatively  simple.  In  the  frequency  domain,  the  equation  for  the 
Gaussian  MTF  is  given  as: 

MTF(f)  =  exp(-bf^)  (in  one  dimension)  (2A) 
MTF(f,,f ')  =  exp  ( -b  (  f  c  +  f  y )  (in  two  dimensions)  (2P) 

f,  f|,  and  f;  denote  spatial  frequency,  typically  in  cycles  pec 
degree  of  visual  angle.  The  rate  parameter,  b,  determines  tne 
fall-off  of  the  curve  across  frequency.  Note  that  there  are  no 
multiplicative  coefficients  preceding  the  exponential  so  that  at  a 
frequency  of  zero,  the  MTF  is  identically  one.  The  effect  of  this 
restriction  is  that  these  filters  do  not  change  the  amount  c^f 
energy  in  the  signal  passing  through  th.e  system,  which  is 
equivalent  to  specifying  that  these  filters  do  not  change  ti'.c 
average  luminance  of  tb.c  images. 

This  representation  of  a  display  MTF  is  prcble.mat  ic . 
Referring  back  to  Figure  J,  neither  display's  MTF  reaches  a  value 
of  unity  at  zero  spatial  frequency.  V.’hen  measuring  the  Michelson 
Contrast  from  a  display,  the  Michelson  Contrast  will  not  be  unity 
at  zero  spatial  frequency  unless  the  lumiinance  measurement  from  the 
dark  screen  is  identically  zero.  Usually,  there  is  somie  amount  of 
ambient  light  in  the  environment  and  even  if  the  display  generates 
no  luminance,  the  ambient  illumination  will  be  reflected  off  of  the 
display.  The  traditional  method  of  circumventing  this  shortcoming 
(i.e.,  the  display  MTF  <  1  at  zero  spatial  frequency)  is  to 
normalize  the  curve.  As  can  be  seen  from  f  igure  1  ,  h.cwcvor .  ir  the 
two  MTFs  are  normalized,  relative  information  about  them  is  lost. 
This  ambiguity  will  not  be  dealt  with  in  the  present  study  but 
should  bo  kept  in  mind  w'hen  comparing  actual  displays. 

In  order  to  filter  the  i)J2  x  512  pixel  images,  the  spatial 
transform  of  the  i  Liters  in  Equations  (.lAj  and  (2R)  were  used  to 
create  convolution  filters  or  kernels  (11  x  11)  in  the  spatial 
do.Tiain.  These  filters  were  numerically  convolved  with  the  inages. 
Tl'iC  spatial  filters  correspenu  i  ng  to  Equations  (2A)  and  (?.E^  are: 

h(x)  (r/b)  '  exp  ( -r?  x'/ b)  (in  on.e  dimension;,  (3A) 
h(x',y)  -  (77/ b)  '  exp  ( -tt' ( x’-ry- )  /  b)  (in  two  dimensions)  (3P) 


where  the  units  of  x  and  y  are  the  inverse  of  f  in  Equations  2A  and 
2B,  typically  degrees  of  visual  angle.  The  five  MTF  filters  used 
in  the  experiment  are  shown  in  Figure  12.  The  top  filter  in  Figure 
12  corresponds  to  an  estimate  of  the  MTF  of  the  IRIS  display  (1000 
vertical  x  1024  horizontal  pixels) .  It  has  been  approximated  by  a 
Gaussian  filter  (MTF  =  exp ( - . 0052X^)  )  .  Although  the  Michelson 
Contrast  or  MTF  at  zero  spatial  frequency  was  actually  .925,  the 
display  MTF  in  Figure  12  has  been  normalized  to  a  value  of  one. 
For  a  more  detailed  explanation  of  the  methods  used  in  estimating 
the  MTF  of  the  IRIS  display,  refer  to  Evans  (1993)  .  The  five 
remaining  MTF  curves  in  Figure  12  correspond  to  a  multiplication  of 
the  top  curve  by  five  curves  obtained  using  Equation  (2A)  with  b 
set  to  .015,  .03,  .05,  .074,  and  .138,  respectively. 

The  reason  for  multiplying  the  experimental  display  MTF  by  the 
"'a*  ■‘matical  MTFs  from  Equation  (2A)  is  a  result  of  the  "double- 
•  ■  problem."  The  image  seen  by  the  observer  in  the  experiment 

' ot  only  filtered  by  the  mathematically  generated  MTF  but  also 
the  display  used  in  the  experiment.  Using  a  linear  systems 
approximation,  the  MTF  of  the  overall  system  is  obtained  by  taking 
the  product  of  the  MTFs  of  the  two  components  in  the  system  (i.e., 
the  simulated  display  and  the  display  used  in  presenting  the 
images)  .  For  a  more  detailed  explanation  of  the  application  of  the 
filters  tc  the  images,  refer  to  Evans  (1993). 

Luminance  Filtering  of  Images 

The  next  step  in  the  development  of  the  filtered  images  was  to 
create  four  levels  of  average  luminance  for  the  images.  Although 
manipulation  of  average  luminance  level  is  a  crude  method  of 
studying  a  factor  which  is  crucial  to  perception  of  the  image, 
there  currently  are  no  better  methods  in  widespread  use. 

Each  of  the  512  x  512  =  262,144  pixels  in  each  image  is 

represented  by  a  DAC  value  between  0  and  255.  Figure  4  shows  the 
distribution  of  the  262,144  pixels  for  each  of  the  five  images  over 
the  255  possible  DAC  levels.  Figure  5,  the  gamma  curve  for  the 
IRIS  monitor  employed  in  the  experiment,  relates  the  DAC  value  of 
the  pixels  to  the  luminance  viewed  on  the  screen  of  the  monitor. 
A  software  program  was  used  to  transform  the  gamma  curve  into  a 
linear  function  such  that  the  doubling  of  a  DAC  vylue  led  to  a 
doubling  of  luminance.  In  this  manner,  it  was  necessary  only  to 
manipulate  DAC  values  in  the  images  in  order  to  perform  linear 
transformations  of  average  luminance. 

By  examining  the  images  (Figs.  7  through  11)  and  comparing 
them  with  their  distributions  in  Figure  4,  a  few  problems  surface 
with  respect  to  luminance,  contrast,  and  image  content.  For 
example,  note  the  pine  image  with  the  bright  sky.  Average 
luminance  in  the  pine  image  is  dominated  on  the  bright  side  by  the 
sky  (DAC  values  of  255) .  The  sky  provides  relatively  little 
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information  which  observers  might  use  in  interpreting  the  image. 
Therefore,  even  though  the  image  might  be  quite  bright,  most  of  the 
information  contained  in  the  image  (e.g.,  the  outline  of  the 
trees) ,  is  contained  in  relatively  dark  areas  of  the  image.  This 
type  of  problem  highlights  the  difficulty  in  describing  images  (or 
information  content)  with  respect  to  variation  of  specific  display 
parameters  (e.g.,  luminance). 

Returning  to  Figure  4,  average  luminance  in  the  experiment  was 
manipulated  by  sliding  the  frequency  distribution  to  the  left  or 
right  across  the  abscissa.  Note,  however,  that  if  the  individual 
distributions  are  moved  to  the  left  or  right,  the  extreme  DAC 
values  0  and  255  present  barriers  at  both  ends  of  the  distribution. 
The  compression  process  was  obtained  through  a  series  of  trial-and- 
error  manipulations.  First,  all  of  the  DAC  values  from  an  image 
were  divided  by  a  constant  to  compress  the  range  of  the  DAC  values. 
Next,  a  constant  was  added  to  each  DAC  value  of  an  image.  The 
additive  constant  for  each  image  was  set  individually  so  that  the 
average  luminance  of  the  images  was  equal.  Figure  13  shows  a 
compression  of  the  DAC  distributions  of  the  five  images  in  order 
that  the  distributions  can  be  moved  across  the  abscissa  to 
manipulate  average  luminance.  Figures  14  through  18  allow 
comparison  of  the  original  images  with  the  compressed  images. 

The  image  compression  process  highlights  the  ambiguity  of 
working  with  image/display/observer  components.  The  compression  of 
the  images,  as  shown  in  Figures  14  through  18,  could  be  a  factor 
associated  with  the  origin  of  the  image  or  a  factor  associated  with 
the  display.  For  experimental  purposes,  we  make  the  assumption 
that  the  compressed  images  represent  the  original  image  content  and 
are  not  some  effect  of  the  display.  Experimental  results,  then, 
are  directly  related  only  to  images  which  have  been  compressed  in 
the  manner  shown  here. 

The  compressed  images  are  visually  distinct  from  the  original 
images.  In  addition,  note  in  Figure  13  that  the  distribution  of 
DAC  values  for  the  Mountain  Scene  is  noticeably  shifted  to  the  left 
relative  to  the  other  images  even  though  the  average  luminance  of 
the  images  is  approximately  equal.  The  spike  at  the  high  end  of 
the  Mountain  Scene  represents  the  sky  in  that  scene  and  the 
presence  of  this  spike  requires  that  the  entire  distribution  be 
shifted  to  the  left  to  meet  the  requirement  of  average  equal 
luminance  across  images.  Although  the  requirement  of  average  equal 
luminance  across  images  is  one  logical  constraint,  other 
constraints  (e.g.,  DAC  distributions  approximately  in  the  same 
range)  may  be  just  as  logical.  These  questions  highlight  the 
problem  of  our  inability  to  characterize  the  information  content  of 
images  within  some  well-specified  domain  and  should  be  kept  in  mind 
in  our  experimental  explorations. 

The  compressed  distributions  shown  in  Figure  13  were  moved 
across  the  abscissa  to  create  four  levels  of  average  luminance. 
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Figure  13 

Compressed  DAC  Distributions  for  Five  Images 


Figure  14 

Original  Versus  Compressed  Airport  Image 


f'jgure  17 

OriginoJ  Versus  Compressed  Ocean  Imago 


approximately  4.0,  6.3,  8.5,  and  11. 1  fL.  Al.  though  it  was 

desirable  to  produce  stimuli  with  average  luminances  greater  than 
il.l  fL,  in  order  to  do  so  it  would  have  been  necessary  to  comipress 
the  original  images  m.ore .  This  artifact  demonstrates  the 
limitations  in  the  range  of  level.s  of  the  independent  nianipulat ion 
due  to  the  experimental  display  device,  not  only  for  lurai.nance 
manipulations  but  for  MTF  manipulations  as  well. 

Figures  19  through  2  3  show  three  of  the  five  experimental 
images  as  filtered  by  the  best,  middle,  and  worst  MTFs  and  two 
luminance  levels  only  .  Due  to  the  photographic  and  pr.\nting 
processes,  these  figures  are  only  approximations  to  the  actual 
experimental  stimuli  used  but  should  give  the  readei  some 
perspective  of  the  effect  of  the  20  filters  on  the  images.  In  the 
following  secticn,  the  methodology  of  the  study  is  prese.-ited  in 
more  detail. 


EXPERIMENTAL  METHODOLOGY 


Observers 

Six  observers,  two  male  and  four  female,  were  employed  as  paid 
suL'icets  in  the  experiment.  All  observers  were  in  their  early  to 
mid-twenties . 

Apparatus 

Figure  24  portrays  the  experimental  setup  used.  An  IRIS 
graphics  wor.kstation  with  a  15-inch  vertical  by  15-inch  horizontal 
display  was  used  to  display  the  two  images.  Each  observer  p-laced 
his/her  chin  in  a  chin  rest  as  shown  in  Figure  24  in  order  to 
sta'cilise  viewing.  The  rionitcr  contained  1,000  vertical  by  1,0.3  5 
,hor i zont.a  1  pixels  and  was  viewed  from  approximately  30  inches  bv 
observers.  .All  spatial  .'■.'r equency'  computations  for  filtcrrng 
poricrTiOd  based  upon  this  30-inch  viewing  distance.  The  e:iT:'’.re 
OLspIay  subtended  ap'prcxir.ately  27.6  decrees  of  '  is’.ial  arg]e  and 
oac.d  pixel  subtended  approximately’  27.6/1024  =  .027  degree.s  or  i  .  62 
minutes  of  arc.  If  we  assr.me  that  the  mask  between  pixels  denotes 
a  dark  space  so  that  the  combination  of  pixel  aT/d  masi:  denotes  .a 
1  igbt-dark  comb  i  na  t  i '"'n  ,  the  theoretical  liiniting  resolution  of  tiio 
dispiay  w’ould  be  1/(.C'/V}  or  approximately  37  eye  ]  es /.deu.nee  of 
vi.sual  angle.  The  top  curve  in  Figure  12  shows  emp.  ir  ica  1  iy 
measured  Michelson  Contrast  for  the  IRIS  monitor  as  a  function  of 
.spatial  frequency  out  to  only  20  cycles  per  degree  of  visual  angle. 

Observers  used  a  mouse  tc  desi.qriate  responses  to  the  p'aired- 
compar  i  sons  preseritatiori .  Baci'.ground  illumination  from  the  room 
was  dark,  less  than  .1  fL. 
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Figure  23 

MTF-Luminance  Filtering  of  the  Pines  Image 
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Procedure 


A  trial  began  with  a  fixation  point  in  the  middle  of  the 
screen.  Approximately  two  seconds  later,  two  512  x  512  images  were 
shown  side-by-side  in  the  middle  of  the  display.  A  small  dark 
patch  running  vertically  in  Llie  middle  of  the  screen  separated  the 
two  images.  Since  there  were  only  1,024  pixels  spaced  horizontally 
across  the  screen,  only  500  pixels  were  presented  horizontally  for 
each  of  the  two  images,  leaving  a  24-pixe.l.  vertical  patch 
separating  the  two  images.  The  background  or  luminance  surrounding 
the  imagery  as  it  was  displayed  was  approximately  7  fL  (CAC  =  128) . 
The  pre-  and  post-trial  luminances  of  the  screen  wiare  less  than  .1 
fL.  Both  the  background  and  pre/post-trial  luminances  were 
considered  important  constants  in  the  experiment  as  the  contrast  of 
the  images  against  the  background  both  in  space  and  time  played  a 
significant  role  in  the  visibility  of  detail  in  the  imagery. 

The  images  remained  visible  on  the  screen  for  approximately 
two  seconds,  following  which  Figure  25  was  displayed  on  the  screen. 
Figure  25  contains  a  rating  scale  denoting  preference  for  the  left 
or  the  right  image  in  the  display.  The  observer  used  the  mouse  to 
slide  an  arrow  across  the  scale  shown  on  the  screen.  When  the 
arrow  reached  the  desired  location  on  the  rating  scale,  the 
observers  pressed  a  button  on  the  mouse  to  denote  their  response. 
A  scale  value  of  zero  denoted  no  preference  but  the  observer  could 
also  use  a  second  button  on  the  mouse  to  denote  that  he/she  thought 
the  left  and  right  images  were  identical.  After  pressing  the 
button  on  the  m.ouse,  the  rating  scale  disappeared  and  the  observer 
initiated  the  next  trial  by  pressing  a  button  on  the  mouse. 

Observers  v;ere  run  through  20  practice  and  400  experimental 
trials  each  session  for  20  sessions,  yielding  8,000  experimental 
trials  per  observer  for  the  entire  study.  Each  of  the  five  image.. 
was  presented  on  8,000/5  =  1,600  trials  so  that  the  20  x  20  --  400 
possible  MTF- luminance  combinations  on  each  trial  were  seen 
1,600/400  =  4  times  for  each  image. 


RESULTS  AND  DISCUSSION 

Data  Compilation:  Preference  Matrices 

As  shown  in  Figure  2  (stimulus  space  in  two  dimensions),  the 
primary  purpose  of  the  study  was  to  generate  a  preference  space  in 
a  third  or  z-dimension  along  with  a  preference  surface  (x,y,z)  in 
the  three-dimensional  space.  For  each  luminance-MTF  combination, 
then,  a  preference  must  be  estimated.  A  traditional  means  of 
estimating  preference  or  scale  values  for  stimuli  in  a 
multidimensional  space  is  based  upon  unfolding  models  which  attempt 
to  satisfy  the  ordering  of  pairwise  preferences  by  placing  the 
points  or  stimuli  in  a  multidimensional  preference  space.  Relative 
distances  from  an  ideal  point  in  the  space,  typically  the  location 
of  the  observer  in  the  space,  represent  preferences  in  the  space. 
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Experimental  Apparatus  Setup 


F  iq 
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In  order  to  use  an  unfolding  approach,  the  pairwise  preference 
probabilities  must  be  calculated.  Preferences  for  the  left  or 
right  image  on  individual  trials  are  cumulated  into  empirical 
preference  matrices  (see  the  matrices  in  the  appendix) .  The  rows 
and  columns  of  the  matrices  in  the  appendix  denote  the  stimuli  used 
in  the  paired  comparison  procedure.  Each  entry  in  the  matrix, 
p(i,j),  denotes  the  probability  of  a  person  preferring  stimulus  i 
to  stimulus  j.  By  symmetry  p(i,j)  =  l-p(j,i),  and  it  was  assumed 
that  p{i,i),  or  the  diagonal  entries  in  the  matrix,  were  .5  or  50%. 

The  appendix  contains  preference  matrices  for  the  present 
study.  The  first  preference  matrix  is  cumulated  over  all  six 
observers  and  five  images.  Each  cell  or  probability  in  this  matrix 
is  based  upon  120  observations.  It  is  also  of  interest  to  know 
whether  individual  observers  differ  in  their  preferences  (i.e.,  the 
third  component  in  Figure  3  (image-display-observer])  and  whether 
the  preferences  change  based  upon  the  image  (i.e.,  the  first 
component  in  Fig.  3).  In  order  to  test  these  hypotheses, 
preference  matrices  were  generated  for  individual  observers 
combined  over  the  five  images  and  for  individual  images  combined 
over  the  six  observers.  The  second  through  sixth  matrices  in  the 
appendix  contain  the  preference  matrices  for  each  of  the  six 
observers  combined  over  the  five  images.  The  probabilities  in  each 
cell  for  these  matrices  are  based  upon  24  observations.  The  final 
five  matrices  in  the  appendix  contain  preference  matrices  for  each 
of  the  five  images  combined  over  the  six  observers.  Probabilities 
in  each  cell  for  these  matrices  were  based  upon  20  observations. 

Preference  matrices  based  upon  only  a  single  observer  and 
single  image  were  also  generated.  Each  probability  in  these 
matrices  was  based  upon  only  four  observations.  These  probability 
estimates  were  determined  to  be  too  unstable  for  use  in  modeling. 
The  next  section  describes  the  modeling  employed  to  unfold  or 
generate  the  preference  estimates  for  the  MTF-luminance 
combination . 

The  Bradlev-Terry-Luce  (BTL)  Model  of  Choice 

The  BTL  Model  (see  De  Soete  and  Carroll,  1992;  or  Bockenholt, 
1992,  for  further  introduction)  is  a  psychological  model  of  choice 
which  can  be  fit  to  preference  probabilities  in  a  paired-comparison 
experiment.  In  the  BTL  Model,  the  probability  that  stimulus  "i"  is 
preferred  to  stimulus  "j"  is  given  as: 

v(i) 

P,/= - •  (4) 

v(i)  +  v(j) 

v(i)  and  v(j)  represent  scale  values  or  preference  strengths  for 
stimulus  "i"  and  stimulus  "j,"  respectively.  In  a  paired- 
comparison  experiment  with  N  stimuli,  the  BTL  model  will  have  (N-1) 
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strength  or  preference  parameters  to  estimate.  A  single  v(#) 
remains  to  serve  as  an  anchor  point.  In  the  present  study,  v(20) , 
the  preference  bcale  value  for  the  20th  MTF-luminance  combination 
(the  worst  MTF  and  the  brightest  luminance) ,  was  set  to  a  value  of 
unity  and  the  remaining  strength  parameters  were  estimated  for  each 
stimulus  or  MTF-luminance  combination  relative  to  this  anchor 
point . 


The  BTL  model  can  be  shown  to  be  conceptually  similar  to  the 
more  popular  model  of  choice,  a  Thurstone  Case  V  Model,  or  a  signal 
detection  model  with  variances  set  to  unity.  By  making  a  monotonic 
transformation,  u  (  i  ) -In  (v  (  i  )  )  or  equiva  lencly ,  e“'"-v(i).  Equation 
(4)  can  be  transformed  into: 


e“"’  1 

p,/  =  l - -  1 - -  l-F[u(i)-u(j)  ]  .  (5) 

0u(O^.gUO) 


F[«]  in  Equation  (5)  is  the  standard  Logistic  Distribution  Function 
with  two  parameters,  a  and  b,  each  set  to  unity.  The  mean  of  the 
Logistic  Distribution  in  Equation  (5)  is  one  and  the  variance  is 
approximately  three  (11^3). 

Figure  26  is  a  plot  of  the  density  function  for  the 
distribution  in  Equation  (5).  Note  the  similarity  of  this  curve  to 
the  normal  density  curve.  The  abscissa  denotes  a  random  variable 
which  is  the  difference  between  a  variable  representing  stimulus 
"i"  and  one  representing  stimulus  "j."  This  conceptual 
representation  is  similar  to  that  of  signal  detection.  On  any 
trial  when  stimulus  "i"  and  "j"  are  paired  together,  a  preference 
or  strength  for  each  image  is  denoted  by  single  random  variable  and 
a  response  is  generated  by  comparing  the  two  random  variables  and 
choosing  the  larger.  The  difference  of  the  two  random  variables  is 
distributed  according  to  Figure  26. 

In  order  to  estimate  the  scale  values,  v(«),  for  a  single 
empirical  preference  matrix,  a  computer  routine  was  used  to 
minimie.e  the  difference  between  the  empirical  preference  matrix  and 
one  generated  using  Equation  (4).  Initial  estimates  of  v(«)  =  1 
were  used  to  generate  an  initial  predict!  v/e  matrix  with  p,/  =  .5. 
Through  iteration  and  estimate  of  partial  derivatives,  the  computer 
routine  sequentially  modified  v(«)  until  it  minimized  the  chi- 
square  difference  between  the  two  matrices: 

n(p,/-p,)' 

X'  =  S - - -  (6) 

i/j  Pu' 
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where  p,j'  denotes  preference  probabilities  from  the  BTL  model  and 
Pjj  are  the  empirically  measured  preference  probabilities.  Table  1 
provides  a  listing  of  all  the  chi-square  fits  performed.  The 
statistical  test  in  each  case  is  based  upon  the  null  hypothesis 
that  the  model  provides  a  reasonable  fit  versus  the  alternative 
hypothesis  that  the  model  is  rejected.  The  20  x  20  matrix  has  (20^- 
20) /2  =  190  independent  data  points  to  fit  since  p„  =  .5  and  p^  = 
P,.  The  BTL  model  has  20-1  =  19  independent  parameters  for  this 
study.  The  degrees  of  freedom  (df)  for  the  chi-square  fit  would 
then  be  190-19  =  171.  For  a  =  .2  and  df  =  171,  the  critical  = 
166.35,  so  none  of  the  model  fits  shown  in  Table  1  could  be 
rejected. 


Table  1 

Chi-Square  Fit  of  The  Bradley-Terry-Luce  Model  to  Empirical  Data 


Data  matrix  Observed  chi-square 

average  over  all  observers  &  images  19.03 
observer  BA  averaged  over  all  images  7.59 
observer  EU  averaged  over  all  images  11.90 
observer  JB  averaged  over  all  images  19.61 
observer  KK  averaged  over  all  images  30.79 
observer  MB  averaged  over  all  images  15.83 
observer  MD  averaged  over  all  images  16.60 
airport  image  averaged  over  all  observers  9.52 
crop  image  averaged  over  all  observers  14.07 
mountain  image  averaged  over  all  observers  6.66 
ocean  image  averaged  over  all  observers  13.00 
pine  image  averaged  over  all  observers  14.57 


Although  the  BTL  model  fit  could  not  be  rejected,  some  models 
are  so  pervasive  that  they  will  fit  any  set  of  data.  To  test  this 
hypothesis,  tl-ie  BTL  model  was  fit  to  a  20  x  20  matrix  containing 
probabilities  generated  using  a  uniform  random  number  generator. 
These  matrices  were  constrained  in  the  same  fashion  as  the 
empirical  preference  matrices  so  that  p,j  =  Pjj  and  p„  =.5.  The  fit 
to  sets  of  randomly  generated  matrices  (using  n  =  20,24,  and  120  as 
shown  in  Equation  (6))  was  consistently  greater  than  600,  large 
enough  to  be  rejected  at  a  =  .01  levels. 

The  scale  values,  v(«),  generated  by  the  BTL  model  fit  for  the 
empirical  matrices  are  shown  in  Table  2.  In  Table  2,  there  are  20 
scale  values  in  each  column  or  for  each  condition.  Each  scale 
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Table  2 


BTL  Scale  Values  for  All  Experimental  Conditions 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

NIF 

LUN 

ALL 

BA_ALL 

£U_ALL 

JBJUL 

KKJIL 

1 

1 

40.023 

1363.191 

22.881 

30.043 

51.235 

1 

2 

137.594 

7039.538 

150.458 

116.083 

171.424 

1 

3 

256.583 

25372.46 

559.019 

382.404 

252.756 

1 

4 

227.986 

69911.03 

761.11 

449.527 

192.117 

2 

1 

13.447 

126.118 

8.353 

10.506 

21.289 

2 

2 

46.102 

$42,666 

56.906 

52.981 

74.075 

2 

3 

69.751 

1656.255 

138.364 

107.359 

120.048 

2 

4 

67.743 

2013.66 

272.658 

106.69 

71.74 

3 

1 

3.738 

12.616 

2.14 

2.732 

7.021 

3 

2 

10.336 

38.584 

11.517 

9.366 

23.273 

3 

3 

16.792 

97.484 

39.836 

19.084 

27.23 

3 

4 

15.271 

189.676 

41.366 

18.324 

18.054 

4 

1 

1.08 

1.39 

0.487 

0.896 

1.93 

4 

2 

2.547 

4.421 

2.704 

2.299 

5.323 

4 

3 

3.646 

11.601 

5.842 

3.314 

5.921 

4 

4 

3.39 

12.781 

8.649 

2.95 

4.041 

( 

1 

0.225 

0.199 

0.05 

0.398 

0.052 

S 

2 

0.359 

0.307 

0.185 

0.492 

0.191 

$ 

3 

0.489 

0.435 

0.435 

0.553 

0.465 

i 

4 

1 

1 

1 

1 

1 

(1) 

MTF  Area  >•  1 

=  6.1 

8 

(2) 

2  =  4.73 

3  =  3.79 

4  =  3.15 

5  =  2.34 


(8) 

(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

n.ALL 

HO.AIL 

AJ«1 

cja 

NJ«.L 

0JM.L 

PJILL 

9.836 

384.816 

142.261 

36.856 

283.495 

23.981 

18.337 

26.263 

802.603 

442.524 

169.196  1271.866 

66.974 

63.014 

27.129 

938.963  1007.9S2 

556.392  3191.456 

116.174 

52.604 

13.624 

620.182 

822.751 

893.558  3448.076 

91.409 

29.856 

4.0H 

98.896 

41.583 

9.074 

50.153 

8.313 

0.822 

11.187 

215.693 

158.517 

50.303 

223.457 

23.216 

24.951 

9.786 

209. 38S 

212.272 

175.216 

466.741 

35.276 

18.609 

7.538 

148.941 

220.018 

223.405 

553.067 

30.904 

12.544 

1.99 

14.072 

9.629 

1.877 

11.018 

2.481 

3.14 

3.547 

29.518 

24.925 

10.127 

33.926 

6.038 

7.092 

3.938 

3S.321 

38.79 

26.394 

61.701 

10.183 

7.838 

2.629 

25.481 

30.183 

44.579 

56.004 

9.469 

4.512 

1.01 

2.888 

1.722 

0.649 

1.774 

0.97 

1.122 

1.279 

$.88 

4.301 

1.719 

4.913 

1.796 

2.697 

1.392 

6.201 

6.443 

4.342 

5.443 

3.059 

2.713 

1.153 

5.242 

5.372 

5.982 

5.607 

2.565 

1.875 

0.549 

0.436 

0.187 

0.189 

0.233 

0.185 

0.364 

0.534 

0.541 

0.32 

0.275 

0.396 

0.307 

0.533 

0.505 

0.524 

0.465 

0.468 

0.44$ 

0.47 

0.618 

1 

1 

1 

1 

1 

1 

1 

Luminance  -1=4.0 

fL 

2  =  6.3 

fL 

3  =  8.5 

fL 

4  =  11. 

1  fL 

(3)  BTL  scale  values  averaged  over  all  observers  and  images 

(4)  BTL  scale  values  for  observer  BA  averaged  over  all  images 

(5)  BTL  scale  values  for  observer  EU  averaged  over  all  images 

(6)  BTL  scale  values  for  observer  JB  averaged  over  all  images 

(7)  BTL  scale  values  for  observer  KK  averaged  over  all  images 

(8)  BTL  scale  values  for  observer  MB  averaged  over  all  images 

(9)  BTL  scale  values  for  observer  MD  averaged  over  all  images 

(10)  BTL  scale  values  for  Airport  Image  averaged  over  all  observers 

(11)  BTL  scale  values  for  Crop  Image  averaged  over  all  observers 

(12)  BTL  scale  values  for  Mountain  Image  averaged  over  all 
observers 

(13)  BTL  scale  values  for  Ocean  Image  averaged  over  all  observers 

(14)  BTL  scale  values  for  Pines  Image  averaged  over  all  observers 
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value  represents  the  preference  for  an  MTF-luminance  combination 
relative  to  the  other  19  MTF-luminance  combinations.  These  scale 
values  complete  the  three-dimensional  preference  space. 
Hypothetically,  a  display  would  be  represented  by  its  MTF-luminance 
combination  or  (x,y)  in  the  plane  of  the  three-dimensional  space 
and  the  predicted  scale  value  (z-axis)  would  locate  the  display  in 
(X, y,z)  . 

From  this  study,  20  points  (5  MTFs  x  4  luminance  levels)  were 
generated  in  the  three-dimensional  space  for  each  of  the  12 
conditions  shown  in  Tables  2  and  3.  Figure  27A  shows  the  20  BTL 
scale  values  for  the  preference  data  averaged  over  observers  and 
images.  Figure  27B  is  a  three-dimensional  plot  of  the  20  points 
where  a  smoothing  algorithm  was  used  to  generate  the  curve  drawn 
through  the  points  from  Figure  27A. 

A  goal  of  this  research  was  to  be  able  to  compare  the  MTF- 
luminance  combinations  from  two  displays  and  predict  which  display 
would  be  preferred.  From  the  curve  in  Figure  27B,  it  would  be 
difficult  to  locate  the  preference  scale  value  in  the  z~dimension 
based  upon  an  MTF-luminance  combination.  A  predictive  equation 
would  be  useful  for  this  purpose.  In  the  following  section, 
regression  is  used  to  generate  surface  equations  for  Figure  27B. 

Generation  of  Predictive  Equations 

Statistical  regression  software  was  used  to  generate 
prediction  equations  for  the  BTL  scale  values  as  a  function  of  MTF 
and  luminance.  Because  the  range  of  scale  values  in  Table  2  was  so 
large,  a  logarithmic  transformation  of  the  data  was  used.  The  form 
of  the  predictive  equations  was: 

log,o  (scale  value  +  1)  =  f(MTF,  Luminance)  (7) 

where  the  goal  was  to  estimate  the  form  of  the  function  f.  From 
fits  of  linear,  quadratic,  and  cubic  polynomials,  quadratic 
formulas  were  chosen  based  upon  their  values.  Table  3  shows  the 
coefficients  and  fits  for  the  12  matrix  conditions  based  upon  the 
following  equation: 

logio  (scale  value  +  1)  =  a  +  b(MTF)  +  c(Lum)  +  d(MTF)(Lum)  (8A) 

H-  e(MTF)^  +  f(Lum)^ 

For  data  averaged  over  all  observers  and  images,  the  prediction  of 
scale  values  is  as  follows: 


scale  value  -  -.oit  (MTn^-.oi6{LUH)^*.o2o{MTin  hum)  (SB) 
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Table  3 


Quadratic  Regression  Coefficients  and  Values  to  the  Equation: 

log,y(scale  value  +  1)  =  a  +  b(MTF)  +  c(Luni)  +  d(MTF)^ 

+  e(Luin)^  +  f  (MTF)  (Lum)  . 


Preference 

condition 

Intercept 

a 

MTF 

b 

LUM 

c 

MTF^ 

d 

LUM^  MTFXLUM 
e  f 

average 

-2 . 51 

.778 

.  226 

-.047 

-.016 

.020 

.983 

observer  BA 

-3.18 

1.033 

.  131* 

-.042 

-.013* 

.  050* 

.991 

observer  EU 

-3.15 

.936 

.253 

-.080 

-.018 

.  044 

.981 

observer  JB 

-2 . 16 

.  610 

.  197* 

-.042 

-.017* 

.  040 

.974 

observer  KK 

-3 . 78 

1.318 

.337 

-.102* 

-.022 

.  013 

.985 

observer  MB 

-1.24 

.300* 

.200 

-.003* 

-.013* 

.  002 

.951 

observer  MD 

-3 . 55 

1.367 

.  243 

-.079* 

-.015 

.  001 

.988 

airport  image 

-3 . 55 

1.266 

.  253 

-.087 

-.018 

.  021 

.986 

crop  image 

-2.57 

.750 

.  181* 

-.054 

-.014 

.  043 

.976 

mountain  image 

-3 .40 

1.163 

.224 

-.073 

-.018 

.  036 

.985 

ocean  image 

-2 . 07 

.  596 

.211 

-.030 

-.014 

.  014 

.983 

pine  image 

-2.20 

.  619 

.306 

-.029 

-.019* 

** 

.971 

*  P  >  .05  (not  significantly  different  from  zero  at  a  =  .05) 
**  <  10’^  and  P  >  .05 


The  preference  conditions  in  Table  3  have  many  commonalities 
with  respect  to  their  regression  coefficients.  For  example,  in 
most  instances  the  linear  slope  attributed  to  MTF  is  two  to  three 
times  the  slope  of  luminance.  The  relationship  across  the  two 
dimensions  (MTF  and  luminance)  and  their  effect  on  preference  was 
the  most  important  effect  of  interest  in  this  study.  Within  the 
MTF  and  luminance  bounds  of  this  experiment,  a  change  in  the  area 
under  the  MTF  curve  has  more  of  an  effect  on  preference  than  an 
equivalent  numerical  change  in  luminance.  It  is  critical,  though, 
that  this  statement  not  be  misinterpreted,  as  it  cannot  be  used  to 
mean  that  changes  in  MTF  are  more  important  than  changes  in 
Luminance  to  viewer  preference.  Suppose,  for  example,  that 
Luminance  had  been  measured  in  tenths  of  a  FootLambert  instead  of 
FootLamberts .  Then,  the  results  here  would  show  that  the  change  in 
Luminance  had  a  much  greater  effect  on  viewer  preference  than  the 
equivalent  numerical  change  in  MTF.  Thus,  across  the  two 
dimensions,  we  can  only  make  statements  such  as  a  change  of  1  unit 
of  MTF  (percent  contrast  X  cycles/degree  of  visual  angle)  is 
equivalent  to  a  change  of  2  units  of  Luminance  (FootLamberts)  with 
respect  to  viewer  preference.  A  numerical  example  given  below  will 
illustrate  the  trade-off  across  these  two  dimensions. 
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SCALING  ESTIMATES  OF  THE  BTL  MODEL 

(PREFERENCE  FOR  EACH  STIMULUS  IS  SHOWN) 


(D  in  K>  CM  T-  o 


(S=lS3a  M=lSyOM)  n3A3T  JiW 
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ire  27A 

a  Averaged  Over  Observers  and  Images 


gure  27B 

t  of  Preference  Scale  Values 


The  linear  effect  (i.e.,  the  parameter  c)  due  to  luminance  was 
also  statistically  insignificant  in  three  instances  as  noted  by 
asterisks  in  Table  3.  Figures  28A-E  show  two-dimensional  cross 
sections  of  preference  as  a  function  of  luminance  for  the  five 
levels  of  MTF  for  each  of  the  six  observers.  Under  the  optimal 
level  of  MTF  (Fig.  28A) ,  preference  is  nonmonotonic  as  a  function 
of  luminance  for  three  of  the  six  observers.  For  three  of  the 
observers,  preference  first  increases  and  then  decreases  as 
luminance  level  increases.  More  detailed  experimental  work  would 
be  required  to  find  the  asymptotic  level  of  luminance.  Most 
likely,  though,  the  asymptotic  level  of  luminance  depends  upon  the 
background  and  pre/post-trial  luminance.  In  this  experiment,  the 
pre/post-trial  luminance  was  quite  dark  (i.e.,  <  .1  fL)  and  the 
background  or  display  luminance  surrounding  the  images  was 
approximately  7  fL.  As  a  result,  the  most  preferred  luminance  was 
probably  in  the.  range  of  8-11  fL,  much  lower  than  if  a  bright 
background  had  been  employed. 

Coefficients  for  squared  terms  in  the  regression  equations 
were  consistently  negative,  denoting  a  slowing  down  or  curvilinear 
effect  of  the  log,(,  of  preference  as  one  or  the  other  of  MTF  and 
luminance  increased.  This  does  not  necessarily  mean  that 
preference  tended  to  asymptote  or  actually  decrease  as  MTF  or 
luminance  independently  increased.  It  implies  that,  within  the 
ranges  of  MTF  and  luminance  studied,  the  rate  of  increase  in  the 
log  of  preference  was  negatively  accelerated  or  slowing  down. 

Finally,  the  interaction  coefficients  (f)  in  Table  3  were  all 
positive.  A  likely  explanation  for  this  finding  is  that  preference 
broke  down  at  low  levels  of  either  MTF  or  luminance  (see  Figs.  28D 
and  E)  .  Note  that  the  scaling  on  the  y-a>;is  in  Figures  28D  and  E 
IS  much  less  than  the  scaling  in  Figures  28A-C.  For  the  two  most 
blurred  MTFs ,  the  amount  of  luminance  mattered  less  because  the 
effect  of  the  MTF  dominated  the  observer's  perception  of  the  image. 
Similarly,  the  lack  of  image  quality  at  the  lowest  luminance  level 
also  dominated  the  observers'  preference  response. 

As  an  example  of  the  use  of  the  Equations  i^  Table  3,  consider 
the  comparison  of  two  displays,  A  and  B.  Using  Equation  (8)  and 
the  regression  coefficients  from  Table  3,  Displays  A  and  B  may  be 
compared  to  one  another  if  they  are  characterized  by  their  MTF- 
luminai^ce  combinations.  Assume  the  two  displays  have  the  following 
characteristics  with  respect  to  their  MTF  areas  and  average 
lum i nances  ( in  f L) : 

MTF,.,  =  4.  MTF,,  =  5. 

Lum„^  =  8  .  Lum, I  =  5  . 


This  case  exemplifies  the  trade-off  problem  across  dimensions  where 
display  A  has  a  lower  or  worse  MTF  but  also  has  more  luminance.  By 
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BTL  PREFERENCES  ACROSS  LUMINANCE 

(FOR  THE  BEST  MF  ONLY) 


Figure  28A 

Preference  as  a  Function  of  Average  Display  Luminance 

(MTF  Area  =6.18) 


BTL  PREFERENCES  ACROSS  LUMINANCE 


(FOR  THE  2ND  BEST  MTF  ONLY) 


LUMINANCE  L£VEL  (1-4) 

□  BA  4-  EU  0  JB  A  KK  X  MB  7  MO 


Figure  28B 

Preference  as  a  Function  of  Average  Display  Luminance 

(MTF  Area  =  4.73) 
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BTL  PREFERENCES  ACROSS  LUMINANCE 


(FOR  T>€  3RO  BEST  IfTF  ONLY) 


LUMINANCE  LEVEL  (1-4) 

□  BA  t  EU  0  JB  A  KK  X  MB  9  MO 


Figure  28C 

Preference  as  a  Function  of  Average  Display  Luminance 

(MTF  Area  =  3.79) 
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BTL  PREFERENCES  ACROSS  LUMINANCE 

(FOR  -mE  4TH  BEST  MF  ONLY) 


□  RA  +  EU 


LUMINANCE  LEVEL  (1-4) 

0  JB  A  KK  X  MB  V  MO 


Figure  28D 

Preference  as  a  Function  of  Average  Display  Luminance 

(MTF  Area  =  3 . ib) 


49 


C  LOO<SCALC  VALUE  ■*-1>  3 


0.32 


BTL  PREFERENCES  ACROSS  LUMINANCE 

(FOR  THE  5TH  BEST  IIF  ONLY) 


Figure  28r' 

l^releronca  as  a  Function  of  Average  Display  Luminance 

(MTF  Area  “  2.34) 
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solving  Equation  (8)  using  the  coefficients  from  the  regression  to 
the  overall  average  in  Table  3,  the  result  obtained  yields  v(A)  = 
19.3  and  v(B)  =  27.8.  Since  the  scale  value  for  display  B  is 
greater  than  that  of  display  A,  the  conclusion  would  be  that 
display  B  is  preferred  to  display  A. 

With  most  image  quality  metrics,  the  comparison  of  the  two 
displays  would  either  end  here  or  anecdotal  evidence  would  be  used 
in  concluding  whether  this  difference  was  significant.  An 
advantage  of  this  empirically  generated  prediction,  though,  is  that 
the  scale  values  can  be  interpreted  through  the  experimental 
paradigm.  Applying  the  scale  values  v(A)  -  19.3  and  v(B)  -  27.8  to 
Equation  (4)  yields; 

v(A)  19.3 

Pai/  - -  - -  =  -41  (9) 

V(A)  +  V(B)  19.3  +  27,8 


or  the  probability  that  display  A  is  chosen  over  display  B  is 
estimated  as  41%.  The  data  used  to  generate  the  scale  values  were 
cumulated  over  observers  and  images.  Generalizing  over  these  two 
facuors,  it  may  be  said  that  if  a  number  of  observers  viewed  a 
number  of  images  on  display  A  and  display  B,  they  would  prefer  the 
images  from  display  A  on  approximately  41%  of  the  trials.  It  may 
be  noted  here  that  the  BTL  preferences  given  here  are  forced-choice 
preferences.  It  is  assumed  now  that  P(„^,  the  probability  that  B  is 
preferred  to  A,  is  59%  and  that  no  indifferent  trials  exist. 

In  order  to  directly  compare  two  displays  based  upon  their 
MTF-luminance  combinations,  Equation  (8)  must  be  used  with  the 
coefficients  from  Table  3.  Equation  (8)  can  be  plotted  as  a 
surface  as  in  Figure  27B  but  it  is  difficult,  if  not  impossible,  to 
visually  locate  the  preference  values  for  MTF-luminance 
combinations . 

Two  displays  may  be  compared  indirectly  in  a  two-dimensional 
graph  by  comparing  both  to  a  reference  display  device.  Figure  29 
is  one  example  of  this,  showing  a  two-dimensional  graph  with 
preference  probability  on  the  y-axis,  MTF  area  on  the  x-axis,  and 
multiple  curves  generated  which  correspond  to  a  range  of  luminance 
levels.  Using  the  previous  example  with  two  displays,  A  and  B,  the 
approximate  preference  probability  of  Display  A  over  the  standard 
is  .45  and  the  approximate  preference  probability  of  Display  B  over 
the  reference  is  .58.  Because  .58  >  .45,  display  B  would  be 
preferred  to  Display  A,  although  the  exact  preference  probability 
of  display  B  over  display  A  could  be  computed  only  by  calculating 
the  actual  BTL  scale  values  of  both  displays  and  using  the  scale 
values  in  Equation  (4)  . 
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The  predictive  validity  of  the  graphs  and  equations  discussed 
in  this  section  apply  only  within  the  ranges  of  MTF  area  (in 
Michelson  Contrast  x  cycles/degree  of  visual  angle)  and  luminance 
(in  f L)  .  The  levels  of  MTF  area  used  in  the  present  study  were 
between  2.34  and  6.18.  From  pilot  work  performed  previously 
(Evans,  1993),  differences  between  the  MTF  areas  become  less 
discriminable  as  magnitude  grows.  Therefore,  extrapolation  of  the 
predictive  equations  beyond  an  MTF  area  of  6.18  would  have  little 
va J idity . 

Average  luminance  levels  of  the  images  in  this  study  ranged 
from  4.0  to  11.1  fL.  Although  preference  for  the  average  luminance 
level  peaked  somewhere  below  11.1  fL,  the  ambient  illumination  (<.l 
fL)  and  CRT  luminance  surrounding  the  images  (approximately  7  fL) 
are  likely  to  be  moderating  factors  in  this  finding. 

Another  problem  in  the  use  and  generality  of  the  predictive 
equiitions  is  a  logical  problem.  When  using  a  display,  if  the 
display  is  capable  of  putting  out  more  luminance,  it  is  logical 
ti'.at  the  di.splay  could  also  produce  lower  luminance  levels.  For 
example,  if  two  displays  are  equivalent  in  the  MTF  dimension  but 
oni.  display  is  capable  of  generating  more  luminance,  it  is  not 
logical  that  the  display  with  less  luminance  capabilities  be 
preferred.  Because  of  this  fact,  the  luminance  value  used  in 
predicting  preference  should  be  the  luminance  less  than  or  equal  to 
the  average  luminance  level  of  the  display  which  maximizes 
preference.  The  scale  value,  then,  .should  be: 

MAX  (v)  over  {MTF,  L  <=  LUM}  (10) 

or  that  we  choose  the  maximum  preference  scale  value  for  the  MTF 
area  of  the  display  and  all  luminances  loss  than  or  equal  to  the 
average  luminance  of  the  display.  Using  the  quadratic  prediction 
in  liquation  (8),  v  can  bo  maximized  with  respect  to  luminance  by 
taking  the  partial  derivative  of  the  function  with  respect  to 
luminance  and  sotting  it  equal  to  zero.  The  result  yields: 

Lmax  -(c  •+  f  (MTF)  )  /  ( 2c)  (11) 

where  MTF  is  the  MTF  area  for  the  particular  display,  Lmax  is  the 
) umi nance  level  which  maximizes  the  preference,  v,  lor  the  MTF 
given,  and  c  ‘•■=  .  220,  c  =>  -.016,  and  f  ^  .02  are  coefficients  taken 
from  the  averaged  r'ogr’cssion  in  Table  3.  Using  the  pi'cvious 
cxamjilo  wlierc  M7T\  =  4,  LUM,^  -  8,  MTF,,  0,  and  LUM,,  -  5,  Lmax  for 
I)  i  ;:f )  1  ayr.  A  and  U  would  be  9.6  and  10.2  1  L,  respectively.  It  the 
avi.'i  age  luminance  ol  disfjlay  A  or  H  was  greater  than  their  Lmax 
value,  tho  luminance  value  of  that  diojilay  would  be  replaced  by 
l.inax  1  c;r  the  [lu rj.'ooc  ol  computing  preference  scale  values.  Figure 
30  is  a  gt  afih  of  Equation  (11)  for  a  range  of  MTF  areas  used  in  the 
study.  Prior  to  using  Itquation  (8)  or  the  grafdi  in  Figure  29  tor 
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Figure  30 

Prediction  of  Luminance  which  Maximizes  Preference  as  a  Function  of  MTF  Area 


predicting  preference,  then,  it  would  be  necessary  to  use  Figure  30 
in  order  to  find  Lmax  first.  As  mentioned,  it  is  suspected  that 
the  predictions  obtained  in  this  section  are  dependent  upon  a 
number  of  moderating  factors  (e.g.,  background  luminance,  field  of 
view) .  Possibly,  one  of  the  most  important  moderating  factors  is 
the  set  of  imagery  used  in  the  study.  Although  there  are  no 
rigorous  or  well-accepted  methods  for  analyzing  the  information 
content  in  imagery,  simple  observation  of  the  images  in  Figures  7 
through  11  would  lead  one  to  conclude  that  the  Ocean  Scene  (Fig. 
10)  has  less  information  than  the  other  images  with  which  to 
develop  preference  standards.  In  addition,  the  Airport  Scene  (Fig. 
7)  has  a  number  of  man-made  features  which  might  hypothetically 
help  in  discriminating  MTF-luminance  combinations.  In  the  next 
section,  statistical  analysis  of  the  effect  of  the  images  on 
preference  is  presented. 

The  Effect  of  Image  on  Preference 

As  emphasized  previously,  none  of  the  results  obtained  in  this 
study  should  be  considered  as  being  independent  of  the  imagery 
being  viewed.  Referring  back  to  Figure  3,  the  information 
contained  in  the  original  image  can  affect  processes  in  the  display 
and  observer  subsystems.  The  five  images  (airport,  crop,  mountain, 
ocean,  and  pines)  chosen  for  viewing  in  this  study  are  visually 
quite  distinct.  They  were  chosen  as  stimuli  in  this  study  because, 
in  a  multidimensional  scaling  study  performed  by  Kleiss  (1992), 
these  stimuli  were  quite  distinct  from  one  another  in  a  two- 
dimensional  similarity  mapping  of  natural  images.  It  was  desirable 
in  this  study  to  determine  if  preference  for  MTF-luminance 
combinations  might  depend  upon  the  image  type  (or  if  preferences 
were  consistent  across  a  diversity  of  image  types) . 

.J^n  analysis  of  covariance  was  performed  on  the  BTL  scale 
values  obtained  for  the  five  images  in  Table  2.  Note  that  the  BTL 
scale  values  in  Table  2  pertaining  to  the  five  images  (airport, 
crop,  mountain,  ocean,  and  pines)  were  averaged  over  observers. 
The  averaging  was  performed  in  order  to  increase  the  number  of 
observations  upon  which  the  preference  probabilities  were  based.  In 
order  that  the  scale  values  for  each  of  the  five  data  sets  in  Table 
2  meet  the  homogeneity  of  variance  assumption,  the  analysis  was 
performed  on  a  logarithmic  transformation  of  the  scale  values. 

The  average  scale  values  for  the  five  images  were  as  follows: 
airport  (159),  crops  (111),  mountain  (484),  ocean  (22),  pines  (13). 
Recall  that  the  scale  values  are  used  in  Equation  (4)  to  obtain  the 
preference  probabilities.  Higher  scale  values  may  be  interpreted 
as  polarization  of  preference  probabilities  towards  zero  and  one. 
This,  in  turn,  may  be  interpreted  as  an  increased  preference 
discrimination  across  the  MTF-luminance  combinations.  From  the 
five  means,  it  can  be  seen  that  there  was  substantially  less 
discrimination  with  the  ocean  and  pines  images  than  the  other 
three.  It  was  suspected  a  priori  that  the  ocean  image  would  not 
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provide  as  much  discriminatory  power  when  viewing  the  MTF- luminance 
combinations,  but  the  low  discriminability  of  the  pines  image  was 
unexpected.  However,  the  DAC  distribution  of  the  pines  image  (see 
Fig.  4)  proved  to  be  more  bimodal  than  the  other  images,  divided 
between  the  brightness  of  the  sky  at  the  top  of  the  image  and  the 
remainder  of  the  image  being  underexposed  (in  the  original  image). 
The  sky  provided  little  information  and  the  remainder  of  the  image 
was  too  underexposed  to  provide  much  discrimination  in  the 
preference  response.  Although  it  was  expected  that  the  airport 
image  with  its  artificial  components  would  yield  the  highest  amount 
of  discriminability,  the  mountain  image  with  its  speckled  pattern 
of  desert  terrain  provided  the  highest  level  of  discrimination. 

The  analysis  of  covariance  is  shown  in  Table  4.  The  factors 
MTF  and  LUMINANCE  in  Table  4  denote  linear  regressions  for  the  two 
factors,  each  based  upon  one  degree  of  freedom.  The  images  were 
shown  to  be  significantly  different  at  p<.0001,  providing  evidence 
that  the  image  can  significantly  affect  the  perception  of  the  MTF- 
Luminance  filters.  The  two  interaction  terms,  MTF  x  IMAGE  and  LUM 
X  IMAGE,  were  both  significant  at  p<.001.  The  interpretation  of 
these  results  is  that  the  slope  of  predicting  the  dependent 
variable  ( log,o  one  plus  the  scale  value)  as  a  function  of  MTF  or 
luminance  changes  significantly  across  images.  Table  5  shows  the 
slopes  for  predicting  the  dependent  variable  (log,o  one  plus  the 
scale  value)  as  a  function  of  either  MTF  or  luminance  for  each  of 
the  five  images. 

As  with  the  linear  coefficients  from  the  quadratic  regression 
in  Table  3,  the  slopes  are  all  positive  and  changes  in  MTF  area 
have  a  larger  effect  on  the  dependent  variable  than  changes  in 
luminance . 

Table  4 

Analysis  of  Covariance  of  BTL  Scale  Values 
for  Determining  Image  Differences 


Source 

DF 

SS 

MS 

F-Value 

sicmif  icanoe 

MTF 

1 

59.08 

59.08 

1175.14 

<.0001 

Luminance 

1 

3 . 59 

3.59 

71.32 

<.0001 

Image 

4 

7.73 

1.93 

38.42 

<.0001 

MTF  X  Image 

4 

4 . 37 

1.09 

21.75 

<.0001 

LUM  X  Image 

4 

1.08 

.27 

5.36 

.0007 

Error 

85 

4.27 

.05 

Total 

99 

80.12 
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Table  5 

Regression  Slopes  for  Dependent  Measure  ( log,o  one  plus  the  scale 
value)  as  a  Function  of  MTF  or  Luminance 


Independent 

Variable 

Imaqe 

MTF 

Luminance 

Airport 

.525 

.  030 

Crops 

.467 

.  059 

Mountains 

,621 

.  040 

Ocean 

.  343 

.  025 

Pines 

.  284 

.  006 

CONCLUSIONS 

The  work  in  this  study  represents  an  empirical  approach  in 
modeling  multidimensional  aspects  of  image/display  quality.  Within 
the  ranges  of  MTF  area  (2.34  -  6.18  in  %  Michelson  Contrast  X 

cycles/degree  of  visual  angle)  and  average  display  luminance 
(4.0  -  11.1  fL)  used  in  the  study,  the  BTL  Model  of  Choice  and 

regression  were  used  to  predict  display  preference  as  a  function  of 
MTF  area  and  average  luminance.  The  results  obtained  from  the 
modeling  allowed  a  quantitative  comparison  across  the  two 
dimensions  of  interest.  Within  the  ranges  of  MTF  area  and  average 
display  luminance  studied,  the  following  general  results  were 
noted: 

(1)  Changes  in  MTF  area  (%  Michelson  Contrast  X  cycles/degree  of 
visual  angle)  had  a  much  greater  effect  on  observer  preference 
than  did  changes  in  average  display  luminance  (in  fL) . 

(2)  Changes  in  viewer  preference  were  monotonical ly  related  to 
changes  in  MTF  area. 

(3)  Changes  in  viewer  preference  were  monotonically  related  to 
changes  in  luminance  (FL)  for  three  observers.  For  the 
remaining  three  observers,  maximum  preference  occurred  at  the 
second  brightest  luminance  level. 


A  more  quantitative  comparison  of  the  relative  effect  of  the  two 
factors  on  preference  was  obtained  using  a  quadratic  regression. 
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Shortcomings  of  the  multidimensional  approach  used  here 
consisted  of  (a)  the  large  number  of  observations  required,  (b)  the 
limits  on  the  ranges  of  factors  studied,  and  (c)  the  difficulty  in 
manipulating  display  parameters. 

With  respect  to  (a)  in  the  present  study,  there  were  20  MTF- 
luminance  combinations  and  the  paired-comparisons  approach  required 
a  total  of  400  possible  stimulus  combinations.  To  generate 
preference  probabilities,  each  of  these  400  possible  combinations 
had  to  be  presented  multiple  times,  enough  to  generate  stable 
probability  estimates. 

With  respect  to  (b) ,  limits  on  the  ranges  of  factors  studied, 
the  modeling  approach  used  in  this  study  was  predicated  on  the  use 
of  a  matrix  of  paired-comparison  preference  probabilities.  To 
unfold  these  preferences  into  a  three-dimensional  space,  all  of  the 
stimuli  must  be  related  directly  or  indirectly  to  all  other 
stimuli.  If  any  stimulus  or  set  of  stimuli  is  not  related  to  ether 
stimuli  (i.e.,  all  preference  probabilities  are  either  zero  or 
one)  ,  then  this  stimulus  or  set  of  stimuli  cannot  be  placed  into 
the  multidimensional  space.  This  restriction  oftentimes  limits  the 
range  of  stimuli  which  may  be  studied.  In  the  case  of  the  present 
study,  the  range  of  luminance  values  and  the  range  of  MTF  areas  was 
restricted. 

With  respect  to  (c)  ,  the  difficulty  in  manipulating  display 
parameters,  it  was  necessary  to  compress  the  original  images  in 
order  to  manipulate  average  luminance.  In  addition,  the  display 
used  in  presenting  the  stimuli  restricted  the  upper  limit  of  MTF 
areas  that  could  be  generated.  The  technique  used  in  this  study  of 
using  a  single  display  to  present  all  of  the  imagery  allowed 
control  of  many  extraneous  factors  that  would  come  into  play  if 
multiple  displays  were  used  to  present  imagery.  The  limitation  of 
this  technique  is  the  restriction  in  the  range  of  the  variables 
studied . 

Finally,  this  study  represents  an  attempt  at  manipulating 
multiple  display  factors  toward  the  purpose  of  comparing  their 
relative  effects  on  image  quality.  In  order  to  compare  the  quality 
provided  by  practical  display  systems,  it  will  be  necessary  to 
quantitatively  compare  trade-offs  across  dimensions  simply  because, 
in  many  instances,  design  of  these  systems  involves  trade-offs  of 
the  display  parameters. 
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APPENDIX  TABLE  2:  PROBABILITY  OF  PREFERRING  ROW  STIMULUS 
OVER  COLUMN  STIMULUS  FOR  AIRPORT  IMAGE 
AVERAGED  OVER  ALL  OBSERVERS 
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APPQfDIX  TKBLE  3:  PROBP^ILITT  OP  PREPERRIHC  ROW  STIMUI^US 
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APPENDIX  TABLE  6:  PROBABILITY  OF  PREPERKING  ROW  STIMULUS 
OVER  COLUMN  STIMULUS  FOR  PINES  IMAGE 
AVERAGED  OVER  ALL  OBSERVERS 
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APP£3fDIX  TABLE  10:  PROBABILITT  OF  PREFERPING  ROW  STIHIH-US 

OVER  COLUKI*  STIKULOS  FOR  OBSERVER  JB 
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APPENDIX  TABLE  12:  PROBABILITY  OF  PREFERRING  ROW  STIMULUS 

OVER  COLUMN  STIMULUS  FOR  OBSERVER  MD 
AVERAGED  OVER  ALL  IMAGES 
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